Stochastic Compositional Gradient Descent: Algorithms for Minimizing Nonlinear Functions of Expected Values

Classical stochastic gradient methods are well suited for minimizing expected-valued objective functions. However, they do not apply to the minimization of a nonlinear function involving expected values, i.e., problems of the form minx f ( Ew[gw(x)] ) . In this paper, we propose a class of stochastic compositional gradient descent (SCGD) algorithms that can be viewed as stochastic versions of quasi-gradient method. SCGD update the solutions based on random sample gradients/subgradients of f, gw and use an auxiliary variable to track the unknown quantity E [gw(x)]. We prove that the SCGD converge almost surely to an optimal solution for convex optimization problems, as long as such a solution exists. The convergence involves the interplay of two iterations with different time scales. For nonsmooth convex problems, the average iterates of SCGD achieve a convergence rate of O(k−1/4) in the general case and O(k−2/3) in the strongly convex case. For smooth convex problems, the SCGD can be accelerated to converge at a rate of O(k−2/7) in the general case and O(k−4/5) in the strongly convex case. For nonconvex optimization problems, we prove that the SCGD converges to a stationary point and provide the convergence rate analysis. Indeed, the stochastic setting where one wants to optimize nonlinear functions of expected values using noisy samples is very common in practice. The proposed SCGD methods may find wide application in learning, estimation, dynamic programming, etc.